5 research outputs found
Trend Detection based Regret Minimization for Bandit Problems
We study a variation of the classical multi-armed bandits problem. In this
problem, the learner has to make a sequence of decisions, picking from a fixed
set of choices. In each round, she receives as feedback only the loss incurred
from the chosen action. Conventionally, this problem has been studied when
losses of the actions are drawn from an unknown distribution or when they are
adversarial. In this paper, we study this problem when the losses of the
actions also satisfy certain structural properties, and especially, do show a
trend structure. When this is true, we show that using \textit{trend
detection}, we can achieve regret of order with
respect to a switching strategy for the version of the problem where a single
action is chosen in each round and when actions
are chosen each round. This guarantee is a significant improvement over the
conventional benchmark. Our approach can, as a framework, be applied in
combination with various well-known bandit algorithms, like Exp3. For both
versions of the problem, we give regret guarantees also for the
\textit{anytime} setting, i.e. when the length of the choice-sequence is not
known in advance. Finally, we pinpoint the advantages of our method by
comparing it to some well-known other strategies
Robust Leader Election in a Fast-Changing World
We consider the problem of electing a leader among nodes in a highly dynamic
network where the adversary has unbounded capacity to insert and remove nodes
(including the leader) from the network and change connectivity at will. We
present a randomized Las Vegas algorithm that (re)elects a leader in O(D\log n)
rounds with high probability, where D is a bound on the dynamic diameter of the
network and n is the maximum number of nodes in the network at any point in
time. We assume a model of broadcast-based communication where a node can send
only 1 message of O(\log n) bits per round and is not aware of the receivers in
advance. Thus, our results also apply to mobile wireless ad-hoc networks,
improving over the optimal (for deterministic algorithms) O(Dn) solution
presented at FOMC 2011. We show that our algorithm is optimal by proving that
any randomized Las Vegas algorithm takes at least omega(D\log n) rounds to
elect a leader with high probability, which shows that our algorithm yields the
best possible (up to constants) termination time.Comment: In Proceedings FOMC 2013, arXiv:1310.459
Tracing Equilibrium in Dynamic Markets via Distributed Adaptation
Competitive equilibrium is a central concept in economics with numerous
applications beyond markets, such as scheduling, fair allocation of goods, or
bandwidth distribution in networks. Computation of competitive equilibria has
received a significant amount of interest in algorithmic game theory, mainly
for the prominent case of Fisher markets. Natural and decentralized processes
like tatonnement and proportional response dynamics (PRD) converge quickly
towards equilibrium in large classes of Fisher markets. Almost all of the
literature assumes that the market is a static environment and that the
parameters of agents and goods do not change over time. In contrast, many large
real-world markets are subject to frequent and dynamic changes. In this paper,
we provide the first provable performance guarantees of discrete-time
tatonnement and PRD in markets that are subject to perturbation over time. We
analyze the prominent class of Fisher markets with CES utilities and quantify
the impact of changes in supplies of goods, budgets of agents, and utility
functions of agents on the convergence of tatonnement to market equilibrium.
Since the equilibrium becomes a dynamic object and will rarely be reached, our
analysis provides bounds expressing the distance to equilibrium that will be
maintained via tatonnement and PRD updates. Our results indicate that in many
cases, tatonnement and PRD follow the equilibrium rather closely and quickly
recover conditions of approximate market clearing. Our approach can be
generalized to analyzing a general class of Lyapunov dynamical systems with
changing system parameters, which might be of independent interest
On bandit learning and pricing in markets
A lot of software systems today need to make real-time decisions to optimize an objective of interest. This could be maximizing the click-through rate of an ad displayed on a web page or profit for an online trading software. The performance of these systems is crucial for the parties involved. Although great progress has been made over the years in understanding such online systems and devising efficient algorithms, a fine-grained analysis and problem specific solutions are often missing. This dissertation focuses on two such specific problems: bandit learning and pricing in gross-substitutes markets.
Bandit learning problems are a prominent class of sequential learning problems with several real-world applications. The classical algorithms proposed for these problems, although optimal in a theoretical sense often tend to overlook model-specific proper- ties. With this as our motivation, we explore several sequential learning models and give efficient algorithms for them. Our approaches, inspired by several classical works, incorporate the model-specific properties to derive better performance bounds.
The second part of the thesis investigates an important class of price update strategies in static markets. Specifically, we investigate the effectiveness of these strategies in terms of the total revenue generated by the sellers and the convergence of the resulting dynamics to market equilibrium. We further extend this study to a class of dynamic markets. Interestingly, in contrast to most prior works on this topic, we demonstrate that these price update dynamics may be interpreted as resulting from revenue optimizing actions of the sellers. No such interpretation was known previously. As a part of this investigation, we also study some specialized forms of no-regret dynamics and prediction techniques for supply estimation. These approaches based on learning algorithms are shown to be particularly effective in dynamic markets